1. Field of the Invention
The present invention relates to a vision sensor capable of omnidirectional observation encompassing a viewing range 360 degrees around the vision sensor. In particular, the present invention relates to an omnidirectional vision sensor which is used in a vision system for a monitoring camera system or a mobile robot, etc., and which can obtain field of view information associated with the entire surroundings in real-time.
2. Description of the Related Art
In recent years, various omnidirectional vision sensors have been proposed as input devices through which visual information covering a broad range is to be input to any device coupled to the omnidirectional vision sensors, with a view to developing applications for monitoring camera systems or mobile robots, etc.
The following techniques are known, for example:
{circle around (1)} a method in which images which are captured by means of a single rotating camera are linked together (Japanese Laid-Open Publication No. 10-105840, entitled xe2x80x9cSystem for automatically detecting an intruding objectxe2x80x9d);
{circle around (2)} a method in which images which are captured by means of a rotating plate mirror are linked together (Japanese Laid-Open Publication No. 11-4373, entitled xe2x80x9cMethod and apparatus for constructing omnidirectional panoramic imagesxe2x80x9d);
{circle around (3)} a method in which omnidirectional images are captured at one time by means of a plurality of fixed cameras (Japanese Laid-Open Publication No. 11-164292, entitled xe2x80x9cImage generation device, image presentation device, image generation method and image synthesis methodxe2x80x9d);
{circle around (4)} a method in which an image from a wide field of view is captured at one time by means of a wide-angle lens such as a fish-eye lens (Japanese Laid-Open Publication No. 11-355763, entitled xe2x80x9cMonitor system and monitor methodxe2x80x9d); and
{circle around (5)} a method in which an image is captured at one time by means of a reflection mirror of a special shape such as a spherical, conical, hyperbolic, etc., shape (Japanese Laid-Open Publication No. 11-218409, entitled xe2x80x9cMethod and apparatus for measuring three-dimensional informationxe2x80x9d).
Method {circle around (1)} mentioned above involves acquiring images of the surroundings by means of a single television camera which is placed on an electrically actuated base and is rotated by 360 degrees, where the images are linked together by image processing. By using this method, it is possible to acquire omnidirectional images with a relatively high resolution. However, since the camera is rotated while acquiring images, it is impossible to acquire omnidirectional images at one time, thus the resultant image is no longer a real-time image.
Method {circle around (2)} mentioned above involves rotating a mirror by 360 degrees so as to acquire images of the surroundings which are reflected by the mirror are captured by means of a fixed camera, where the images are linked together by image processing. Thus, it is possible to acquire omnidirectional images with a relatively high resolution, as is the case with method {circle around (1)}. However, since the mirror is rotated while acquiring images, it is impossible to acquire omnidirectional images at one time, thus resultant image is no longer a real-time image, as is the case with method {circle around (1)}.
Methods {circle around (1)} and {circle around (2)} mentioned above utilize a mechanical means for rotating a camera or a mirror, respectively, thus requiring some sort of maintenance work for the mechanical means in order to enable operation over a long period of time. Accordingly, methods {circle around (3)} to {circle around (5)} mentioned above have been proposed as methods which enable a one-time acquisition of omnidirectional images without employing any mechanical means.
Method {circle around (3)} mentioned above involves acquiring omnidirectional images at one time by employing a plurality of fixed cameras, and is advantageous from the perspective of obtaining images in real-time. Moreover, since no special mechanical means is required, this method is suitable for long periods of use, and provides for good reliability. However, there is a problem in that the use of a plurality of camera leads to an increased system cost.
Methods {circle around (4)} and {circle around (5)} mentioned above employ a wide-angle lens or a reflection mirror of a specific shape to enable a one-time acquisition of an image from a wide field of view. As is the case with method {circle around (3)} mentioned above, this method is advantageous from the perspective of obtaining images in real-time, and, since no special mechanical means is required, this method is suitable for long periods of use and provides for good reliability. Furthermore, unlike method {circle around (3)} mentioned above, only one camera is required, thereby reducing the system cost. However, with methods {circle around (4)} and {circle around (5)}, it is impossible to acquire complete omnidirectional images encompassing 360 degrees. In other words, the resultant field of view includes a blind spot(s).
Hereinafter, the field of view and blind spots which are inherent in methods {circle around (4)} and {circle around (5)} mentioned above will be described with reference to FIGS. 6 to 10. Each plane of FIGS. 6 to 10 is a vertical plane which contains a central axis therein, with a camera being disposed below a lens or a mirror.
FIG. 6 illustrates a field of view in the case where a wide-angle lens 10 is employed in method {circle around (4)} mentioned above. When the system is constructed so that the wide-angle lens 10 is disposed with its convex portion xe2x80x9cupxe2x80x9d (as shown in FIG. 6), with the imaging means including a camera being located below the wide-angle lens 10, it would be possible to acquire an image from the space above a horizontal plane extending 360 degrees around the lens, an image of only an upper half of the surrounding sphere along the vertical direction can be captured. That is, the lower half of the surrounding sphere is left as a blind spot.
FIG. 7 illustrates a field of view in the case where a conical mirror 20 is employed as a body-of-revolution mirror in method {circle around (5)} mentioned above. While the images captured by this method encompass a horizontal span covering 360 degrees around the mirror, the mirror face presents an obstacle along the vertical direction, creating a blind spot above and below the horizontal span. In other words, a blind spot exists in the xe2x80x9cfrontxe2x80x9d of the camera (imaging means).
FIG. 8 illustrates a field of view in the case where a spherical mirror 30 is employed as a body-of-revolution mirror in method {circle around (5)} mentioned above. While the images captured by this method encompass a horizontal span covering 360 degrees around the mirror, the mirror face presents an obstacle along the vertical direction, creating a blind spot above the horizontal span. In other words, a blind spot exists in the xe2x80x9cfrontxe2x80x9d of the camera (imaging means).
FIG. 9 illustrates a field of view in the case where a hyperbolic mirror 40 is employed as a body-of-revolution mirror in method {circle around (5)} mentioned above. While the images captured by this method encompass a horizontal span covering 360 degrees around the mirror, the mirror face presents an obstacle along the vertical direction, creating a blind spot above the horizontal span. In other words, a blind spot exists in the xe2x80x9cfrontxe2x80x9d of the camera (imaging means).
FIG. 10 illustrates a field of view in the case where a parabolic mirror 50 is employed as a body-of-revolution mirror in method {circle around (5)} mentioned above. While the images captured by this method encompass a horizontal span covering 360 degrees around the mirror, the mirror face presents an obstacle along the vertical direction, creating a blind spot above the horizontal span. In other words, a blind spot exists in the xe2x80x9cfrontxe2x80x9d of the camera (imaging means).
Thus, according to any of methods {circle around (4)} and {circle around (5)} mentioned above, blind spots exist in a portion of the field of view. Method {circle around (4)} is also disadvantageous in that the resultant field of view only expands in the upper direction, as described above. Therefore, when method {circle around (4)} is implemented in a mobile robot, for example, only the ceiling of a building which accommodates the mobile robot would always be observed. Thus, method {circle around (4)} does not enable sufficient observation of the lateral directions of the robot, which is essential in preventing the robot from colliding with other objects, while only permitting the observation of the upper region which does not require much attention.
In recent years, method {circle around (5)} mentioned above, which involves the use of a body-of-revolution mirror, has attracted much attention because this method is advantageous from the perspective of obtaining images in real-time, low cost, and high reliability, in spite of some blind spots in a portion of the field of view. In particular, when method {circle around (5)} is implemented with a hyperbolic mirror (among other bodies-of-revolution), an optical system of a perspective projection type is typically used. As a result, the obtained image can be easily converted to an image as seen from a focal point of the mirror (which should appear similar to an image which is imaged via a common camera), or an image which would be obtained by rotating a camera along a vertical axis (a cylindrical omnidirectional image). Thus, a greater variety of image processing is possible than any other method which employs mirrors. An omnidirectional visual system employing a hyperbolic mirror is described in Japanese Laid-Open Publication No. 6-295333.
However, in any variant of method {circle around (5)} mentioned above, blind spots exist in the frontal direction of the camera (imaging means). In order to be able to apply omnidirectional vision sensors to a wide range of usage, it would be desirable to further reduce the blind spots.
According to the present invention, there is provided an omnidirectional vision sensor comprising: an optical system including a body-of-revolution mirror having a convex portion and having a symmetrical structure with respect to a revolution axis, wherein the body-of-revolution mirror includes a cutaway section in the convex portion of the body-of-revolution mirror so as to allow light incident from surroundings of the revolution axis of the body-of-revolution mirror to be collected; and imaging means, including a light-receiving element for receiving the collected light and image processing means for converting an optical image generated from the collected light received by the light-receiving element into image data, wherein the revolution axis of the body-of-revolution mirror and an optic axis of the light-receiving element coincide.
According to the above structure of the present invention, an area in the frontal direction of an imaging means (i.e., an upper direction of the optical system), which would be a blind spot in conventional structures, also becomes part of the field of view. Thus, according to the present invention, the field of view is expanded, as will be described more specifically in an embodiment to follow.
In one embodiment of the present invention, the optical system further comprises a wide-angle lens provided in the cutaway section of the body-of-revolution mirror, the wide-angle lens being disposed so that a convex portion of the wide-angle lens faces away from the imaging means.
According to the above structure of the present invention, the field of view can be expanded based on a further reduction of the blind spot, as will be described more specifically in an embodiment to follow.
In another embodiment of the present invention, a field of view of the wide-angle lens coincides with a blind spot of the body-of-revolution mirror.
According to the above structure of the present invention, the blind spot in the upper direction can be eliminated, as will be described more specifically in an embodiment to follow.
Thus, the invention described herein makes possible the advantages of providing an omnidirectional vision sensor which can be used in a wide range of applications, with substantially reduced blind spots compared with those associated with conventional omnidirectional vision sensors employing a body-of-revolution mirror (e.g., a conical mirror, a spherical mirror, a hyperbolic mirror, or a parabolic mirror).
This and other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description with reference to the accompanying figures.